CLAIMS 



1. A method of associating additional 
information with a video including a 
plurality of frames comprising: 

(a) identifying at least one of said frames; 

(b) providing a descriptive stream separate 
from said video; 

(c) including said additional information in 
said descriptive stream related to said at 
least one of said frames; 

(d) providing saidj^ideg for displaying on a 
display; and 

(e) selectively providing said additional 
information to a viewer approximately the 
time of said providing said video. 

2. The method of claim 1 wherein said 
additional information includes at least one of an object 
index, a textual description, a voice annotation, an 
image feature, an object link, a URL link, and a Java 
applet . 

3. The method of claim 1 wherein said 
identifying is an object within said frame. 

4. The method of claim 1 where said 
descriptive stream is related to a plurality of said 
frames . 



5. The method of claim 4 wherein said at 
least one of said frames are in sequential order in said 
video . 
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6. The method of claim 4 wherein said at 
least one of said frames are in nonsequential order in 
said video. 

5 7. A method of claim 3 wherein said 

additional information is related to said object. 

8. The method of claim 1 wherein said 
descriptive stream includes an index synchronizing said 
10 video with said descriptive stream. 



9. The method of claim 1 wherein said 
descriptive stream includes copyright information. 

15 10. The method of claim 1 wherein said 

descriptive stream is encoded separately from said video. 

11. The method of claim 10 wherein said video 
is decoded in the same manner independently of whether 

20 said descriptive stream is provided. 

12. The method of claim 11 wherein said video 
is at least one of MPEG-2 and television broadcast 
format . 

25 

13. The method of claim 1 wherein said 
additional information is presented to said viewer on a 
remote control. 



30 



14. The method of claim 1 wherein an audible 
signal indicates the availability of said additional 
information . 
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15. The method of claim 1 wherein a visual 
signal indicates the availability of said additional 
information . 

16. The method of claim 7 wherein said 
additional information includes textual based information 
related to said object. 

17. The method of claim 7 wherein said 
additional information includes audible information 
related to said object. 

18. The method of claim 7 wherein said 
additional information includes image features comprising 
at least one of texture, shape, dominant color, and a 
motion model related to said object. 

19. The method of claim 7 wherein said 
additional information includes links to at least one of 
other objects and frames within said video. 

20. The method of claim 7 wherein said 
additional information includes program instructions 
related to said object. 

21. A video system comprising: 

(a) an encoder that includes additional 
information within a video stream 
including a video including a plurality of 
frames, where said additional information 
is related to at least one of said frames; 

(b) a receiver that receives said video and 
said additional information, and said 
receiver decodes said video in the same 
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manner independently of whether said 
additional information is provided; 

(c) a display for displaying said video; and 

(d) a trigger mechanism for selectively 

5 presenting said additional information to 

a viewer at approximately the time of 
presenting said frames to said viewer. 

22. The system of claim 21, further 
10 comprising: 

(a) a transmitter for transmitting said video 
signal and said additional information; 
and 

(b) a receiver for receiving said video signal 
15 and said additional information. 

23. The system of claim 22 wherein said 
encoder is at least one of a video camera and a computer. 

20 24. The system of claim 21 wherein said 

trigger mechanism is located in a remote control device. 

25. The system of claim 21 wherein said 
additional information is provided by a remote control 

25 device. 

26. The method of claim 21 wherein said 
additional information is related to an object within 
said frame and includes links to at least one of other 

30 objects and frames within said video. 

27. The method of claim 21 wherein said 
additional information is related to an object within 
said frame and includes program instructions related to 

35 said object. 




additional 
said frame 
to said obj 

5 

29. The method of claim 21 wherein said 
additional information is related to an object within 
said frame and includes audible information related to 
said object. 

10 

30. The method of claim 21 wherein said 
additional information is related to an object within 
said frame and includes image features comprising at 
least one of texture, shape, dominant color, and a motion 

15 model related to said object. 

31 

comprising : 

(a) 

20 

(b) 



25 

(c) 

said additional information to a viewer in 
response to selecting said object . 
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28. The method of claim 21 wherein said 
information is related to an object within 
and includes textual based information related 
ect . 



. A system for presenting information 

a unitary file containing an image and 
. additional information associated with 
said image; 

a selection mechanism that permits the 
selection of objects in said image for 
which said additional information is 
related thereto; and 

a presentation mechanism that provides 
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32 . The system of claim 31 wherein said file 
includes said image followed by said additional 
information . 



10 
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33. The system of claim 32 wherein said image 
and said additional information are separated by a marker 
indicating the end of said image. 

34. The system of claim 33 wherein an image 
viewer which does not recognize said additional 
information will display said image properly and 
recognize said marker as indicating the end of said 
image . 

35. The system of claim 34 wherein said image 
is in a JPEG format. 



36. The system of claim 31 wherein said 
15 additional information is organized in at least two 
layers comprising: 

(a) a first layer containing information 

describing the location of objects within 
said image; and 
20 (b) a second layer containing additional 

information regarding said objects within 
said image, where said first layer 
contains fewer bytes than said second 
layer. 



25 



37. The system of claim 36 wherein said second 
layer follows said first layer, which in turn follows 
said image file. 



30 



38. The system of claim 36 wherein said first 
layer contains a length identifier describing the length 
of said first layer. 
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39. The system of claim 36 wherein said first 
layer contains a number of objects identifier describing 
the number of objects identified by said first layer. 

5 40. The system of claim 36 wherein said first 

layer contains a number of data identifier describing the 
number of data items associated with a particular said 
object . 

10 41. The system of claim 36 wherein said first 

layer contains a first definition of the outline of an 
object of said image. 

42. The system of claim 36 wherein said second 
15 layer contains a length identifier describing the length 

of said second layer. 

43. The system of claim 36 wherein said second 
layer contains an array of offsets that identify the 

20 start of each data item. 

44. The system of claim 41 wherein said second 
layer contains a second definition of the outline of said 
object of said image, where said second definition more 

25 closely approximates the outline of said object than said 
first definition. 

45. The system of claim 41 wherein said second 
layer contains a second definition of the outline of said 

30 object of said image, where said second definition 
contains more bytes than said first definition. 

46. The system of claim 36 wherein said second 
layer includes sound data related to said object. 

35 



• 



10 
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47. The system of claim 36 wherein said second 
layer includes HTML meta tags related to said object. 

48. The system of claim 36 wherein said second 
layer includes textual annotations related to said 
object . 

49. The system of claim 36 wherein said second 
layer includes an HTML page to be rendered. 

50. The system of claim 36 wherein said second 
layer includes a Java Applet related to said object. 

51. The system of claim 36 wherein said second 
15 layer includes a color histogram. 

52. The system of claim 36 wherein said second 
layer includes data related to the conditions under which 
said image was created including at least one of 

20 lighting, camera settings, and time of acquisition. 

53. The system of claim 36 wherein said second 
layer includes data related to information for 
reproducing said image including at least one of cropping 

25 information, paper type, camera settings, and image 
production settings . 

54. The system of claim 36 wherein said second 
layer includes another image to be superimposed upon said 

3 0 image. 



55. The system of claim 36 wherein said second 
layer includes data regarding the author of said image. 
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56. The system of claim 36 wherein said second 
layer includes copyright data regarding the copyright of 
said image. 



57. The system of claim 56 said copyright data 

is encoded. 

58. The system of claim 36 wherein said second 
layer includes information regarding how said image 
should be viewed. 



59. A method of presenting information 

comprising : 

(a) displaying an image from a unitary file 
containing an image and additional 
information associated with said image; 

(b) selecting an object in said image for 
which said additional information is 
related thereto; and 

(c) providing said additional information to a 
viewer in response to selecting said 
object . 



60. The method of claim 59 wherein said file 
includes said image followed by said additional 
information . 

61. The method of claim 60 wherein said image 
and said additional information are separated by a marker 
indicating the end of said image . 



62. The method of claim 61 wherein an image 
viewer which does not recognize said additional 
information will display said image properly and 
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recognize said marker as indicating the end of said 



image . 



63. The method of claim 62 wherein said image 



5 



is in a JPEG format. 



64. The method of claim 59 wherein said 



additional information is organized in at least two 
layers comprising a first layer containing information 
10 describing the location of objects within said image and 
a second layer containing additional information 
regarding said objects within said image, where said 
first layer contains fewer bytes than said second layer. 

15 65. The method of claim 64 wherein said second 

layer follows said first layer, which in turn follows 
said image file. 

66. The method of claim 64 wherein said first 
20 layer contains a length identifier describing the length 

of said first layer. 

67. The method of claim 64 wherein said first 
layer contains a number of objects identifier describing 

25 the number of objects identified by said first layer. 

68. The method of claim 64 wherein said first 
layer contains a number of data identifier describing the 
number of data items associated with a particular said 

30 object. 

69. The method of claim 64 wherein said first 
layer contains a first definition of the outline of an 
object of said image. 



35 
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70. The method of claim 64 wherein said second 
layer contains a length identifier describing the length 
of said second layer. 

71. The method of claim 64 wherein said second 
layer contains an array of offsets that identify the 
start of each data item. 



72. The method of claim 69 wherein said second 
10 layer contains a second definition of the outline of said 
object of said image, where said second definition more 
closely approximates the outline of said object than said 
first definition. 



15 73. The method of claim 69 wherein said second 

layer contains a second definition of the outline of said 
object of said image, where said second definition 
contains more bytes than said first definition. 

20 74. The method of claim 64 wherein said second 

layer includes sound data related to said object. 



75. The method of claim 64 wherein said second 
layer includes HTML meta tags related to said object. 

25 

76. The method of claim 64 wherein said second 
layer includes textual annotations related to said 
object. 

30 77. The method of claim 64 wherein said second 

layer includes an HTML page to be rendered. 
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78. The method of claim 64 wherein said second 
layer includes a Java Applet related to said object. 
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79. The method of claim 64 wherein said second 
layer includes a color histogram. 

80. The method of claim 64 wherein said second 
5 layer includes data related to the conditions under which 

said image was created including at least one of 
lighting, camera settings, and time of acquisition. 

81. The method of claim 64 wherein said second 
10 layer includes data related to information for 

reproducing said image including at least one of cropping 
information, paper type, camera settings, and image 
production settings . 

15 82. The method of claim 64 wherein said second 

layer includes another image to be superimposed upon said 
image. 

83. The method of claim 64 wherein said second 
20 layer includes data regarding the author of said image. 

84. The method of claim 64 wherein said second 
layer includes copyright data regarding the copyright of 
said image. 

25 

85. The method of claim 64 said copyright data 

is encoded. 

86. The method of claim 64 wherein said second 
30 layer includes information regarding how said image 

should be viewed. 



35 



87. A method for creating an image file 

comprising: 

(a) providing an image; 
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(b) selecting an object in said image; 

(c) providing additional information related 
to said object; and . .. 

(d) storing said image and said additional 
information in a unitary file. 
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88. The method of claim 87 wherein said file 
includes said image followed by said additional 
information . 

89. The method of claim 88 wherein said image 
and said additional information are separated by a marker 
indicating the end of said image. 



15 90. The method of claim 89 wherein an image 

viewer which does not recognize said additional 
information will display said image properly and 
recognize said marker as indicating the end of said 
image. 



20 



91. The method of claim 90 wherein said image 
is in a JPEG format. 



92. The method of claim 87 wherein said 
25 additional information is organized in at least two 

layers comprising a first layer containing information 
describing the location of objects within said image and 
a second layer containing additional information 
regarding said objects within said image, where said 
30 first layer contains fewer bytes than said second layer. 

93. The method of claim 92 wherein said second 
layer follows said first layer, which in turn follows 
said image file. 



35 
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94. The method of claim 92 wherein said first 
layer contains a length identifier describing the length 
of said first layer. 

5 95. The method of claim 92 wherein said first 

layer contains a number of objects identifier describing 
the number of objects identified by said first layer. 

96. The method of claim 92 wherein said first 
10 layer contains a number of data identifier describing the 

number of data items associated with a particular said 
object . 

97. The method of claim 92 wherein said first 
15 layer contains a first definition of the outline of an 

object of said image. 

98. The method of claim 92 wherein said second 
layer contains a length identifier describing the length 

20 of said second layer. 

99. The method of claim 92 wherein said second 
layer contains an array of offsets that identify the 
start of each data item. 

25 

100. The method of claim 97 wherein said second 
layer contains a second definition of the outline of said 
object of said image, where said second definition more 
closely approximates the outline of said object than said 

30 first definition. 

101. The method of claim 97 wherein said second 
layer contains a second definition of the outline of said 
object of said image, where said second definition 

35 contains more bytes than said first definition. 
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102. The method of claim 92 wherein said second 
layer includes sound data related to said object . 

103. The method of claim 92 wherein said second 
5 layer includes HTML meta tags related to said object. 

104. The method of claim 92 wherein said second 
layer includes textual annotations related to said 

ob j ect . 

10 

105. The method of claim 92 wherein said second 
layer includes an HTML page to be rendered. 

106. The method of claim 92 wherein said second 
15 layer includes a Java Applet related to said object. 

107. The method of cla im 92 wherein said second 
layer includes a color histogram. 

20 108. The method of claim 92 wherein said second 

layer includes data related to the conditions under which 
said image was created including at least one of 
lighting, camera settings, and time of acquisition. 

25 109. The method of claim 92 wherein said second 

layer includes data related to information for 
reproducing said image including at least one of cropping 
information, paper type, camera settings, and image 
production settings . 



30 



110. The method of claim 92 wherein said second 
layer includes another image to be superimposed upon said 
image . 
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111. The method of claim 92 wherein said second 
layer includes data regarding the author of said image. 

112. The method of claim 92 wherein said second 
layer includes copyright data regarding the copyright of 
said image. 

113. The method of claim 92 said copyright data 
is encoded. 

114. The method of claim 92 wherein said second 
layer includes information regarding how said image 
should be viewed. 

115. The method of claim 92 wherein said first 
layer is transmitted from a first computer to a second 
computer together with said image. 

116. The method of claim 115 wherein portions 
of said second layer are transmitted from said first 
computer to said second computer upon request by said 
first computer. 



117. The method of claim 116 wherein said 
request is in response to a user selecting an object 
within said image. 



